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Abstract 

The  Technology  Readiness  Level  (TRL)  scale  is  a  measure  of  maturity  of  an  individual 
technology,  with  a  view  towards  operational  use  in  a  system  context.  A  comprehensive  set  of 
concerns  becomes  relevant  when  this  metric  is  abstracted  from  an  individual  technology  to  a 
system  context,  which  may  involve  interplay  among  multiple  technologies  that  are  integrated 
through  the  defense  acquisition  process.  This  paper  proposes  the  development  of  a  system- 
focused  approach  for  managing  system  development  and  making  effective  and  efficient 
decisions  during  the  defense  acquisition  process.  For  this  to  be  accomplished,  a  new  System 
Readiness  Level  (SRL)  index  will  incorporate  both  the  current  TRL  scale  and  the  concept  of  an 
integration  readiness  level  (IRL).  This  paper  describes  the  foundations  for  the  SRL  and  provides 
techniques  for  determining  current  and  future  readiness  of  a  system  to  determine  its  position  in 
the  defense  acquisition  process.  In  addition,  it  proposes  optimization  models  than  can  provide 
management  with  an  optimal  development  plan  that  can  meet  the  objectives  of  the  development 
team,  based  on  constrained  resources.  These,  in  turn,  can  become  the  foundation  for  the 
development  of  a  monitoring  and  evaluation  tool  that  will  be  analogous  to  Earned  Value 
Management  used  in  project  management. 

1.  Introduction 

In  theory,  technology  and  system  development  follow  similar  evolution  (or  maturation) 
paths;  a  technology  is  inserted  into  a  system  (e.g.,  spiral  development)  based  on  its  maturity, 
functionality  and  environmental  readiness,  and  ability  to  interoperate  with  the  intended  system. 
However,  the  assessments  made  during  the  acquisition  lifecycle  that  support  these  decisions 
are  not  always  effective,  efficient,  or  well  developed.  Recently,  the  Government  Accounting 
Office  (GAO)  stated  that  many  of  the  programs  in  the  Department  of  Defense  (DoD)  plan  to  hold 
design  reviews  or  to  make  a  production  decision  without  demonstrating  the  level  of  technology 
maturity  that  should  have  been  there  before  the  start  of  development  (GAO,  1999).  In  many  US 
government  agencies  and  contractors.  Technology  Readiness  Level  (TRL)  is  used  to  assess 
the  maturity  of  evolving  technologies  (materials,  components,  devices,  etc.)  prior  to 
incorporating  a  technology  into  a  system  or  subsystem.  In  the  1990s  the  National  Aeronautics 
and  Space  Administration  (NASA)  instituted  this  nine-level  metric  as  a  systematic 
metric/measurement  approach  to  assess  the  maturity  of  a  particular  technology  and  to  allow 
consistent  comparison  of  maturity  between  different  types  of  technologies  (Mankins,  2002). 
Given  the  pragmatic  benefits  of  this  concept,  in  1999,  the  DoD  embraced  a  similar  TRL  concept 
(USD(AT&L),  2005:  DoD,  2005).  While  the  use  of  TRL  is  similar  in  these  organizations,  TRL 
was  not  intended  to  measure  the  integration  of  technologies,  but  was  to  be  used  as  ontology  for 
contracting  support  (Sadin,  Povinelli,  &  Rosen,  1989),  thus  TRL  does  not  address: 

■  A  complete  representation  of  the  (difficulty  of)  integration  of  the  subject  technology  or 
subsystems  into  an  operational  system  (Mankins,  2002;  Dowling  &  Pardoe,  2005;  Valerdi 
&  Kohl,  2004), 

■  The  uncertainty  that  may  be  expected  in  moving  through  the  maturation  of  TRL  (Mankins, 
2002;  Dowling  &  Pardoe,  2005;  Smith,  2005;  Cundiff,  2002),  and 

■  Comparative  analysis  techniques  for  alternative  TRLs  (Mankins,  2002;  Dowling  &  Pardoe, 
2005;  Smith,  2005;  Cundiff,  2002). 
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Based  on  these  fundamental  conjectures,  a  more  comprehensive  set  of  concerns 
becomes  relevant  when  TRL  is  abstracted  from  the  level  of  an  individual  technology  to  a  system 
context,  which  usually  involves  the  interplay  among  multiple  technologies.  Similarly  relevant  is 
the  case  in  which  these  technologies  are  integrated  through  the  defense  acquisition  process. 
That  is,  component  level  considerations  relating  to  integration,  interoperability,  and  sustainment 
become  equally  or  more  important  from  a  systems  perspective  in  an  operational  environment. 

Technology  insertion  as  part  of  a  defense  acquisition  process  needs  a  quantitative 
assessment  tool  that  can  determine  whether  a  group  of  separate  technology  components  with 
their  associated  (and  demonstrated)  TRL  ratings  can  be  integrated  into  a  larger  complex  system 
at  a  reasonably  low  risk  in  order  to  perform  a  required  function  or  mission  at  some  performance 
level. 


However,  before  such  a  tool  can  be  developed,  we  must  first  address  the  issue  of 
measuring  the  maturity  of  the  integration  elements.  The  very  first  attempt  to  address  this  was 
done  by  Mankins  (2002)  when  he  proposed  an  Integrated  Technology  Analysis  Methodology  to 
estimate  an  Integrated  Technology  Index  (ITI).  The  ITI  was  then  used  for  a  comparative  ranking 
of  competing  advanced  systems.  The  study  brought  to  the  forefront  the  difficulty  of  progressing 
through  the  TRL  index  and  choosing  between  competing  alternative  technologies;  it  did  not 
adequately  address  the  integration  aspects  of  systems  development. 

Based  on  concerns  for  successful  insertion  of  technologies  into  a  system,  the  Ministry  of 
Defence  in  the  United  Kingdom  developed  a  Technology  Insertion  Metric  that  includes,  among 
other  things  an  Integration  Maturity  Level  (Dowling  &  Pardoe,  2005).  Building  upon  these 
efforts,  Gove,  Sauser,  and  Ramirez-Marquez  (2008)  performed  a  thorough  review  of  aerospace 
and  defense-related  literature  to  identify  the  requirements  for  developing  a  seven-level 
integration  metric  which  they  called  Integration  Readiness  Level  (IRL).  It  has  since  evolved  into 
the  nine-level  concept  (Gove,  2007)  described  in  Table  1  below. 


Table  1.  Integration  Readiness  Levels 


IRL 

Definition 

9 

Integration  is  Mission  Proven  through  successful  mission  operations. 

8 

Actual  integration  is  completed  and  Mission  Qualified  through  test  and 
demonstration,  in  the  system  environment. 

7 

The  integration  of  technologies  has  been  Verified  and  Vaiidated  with 
sufficient  detail  to  be  actionable. 

6 

The  integrating  technologies  can  Accept,  Translate,  and  Structure 
Information  for  its  intended  application. 

5 

There  is  sufficient  Control  between  technologies  necessary  to  establish, 
manage,  and  terminate  the  integration. 

4 

There  is  sufficient  detaii  in  the  Quality  and  Assurance  of  the  integration 
between  technoiogies. 

3 

There  is  Compatibility  (i.e.,  common  language)  between  technologies  to 
orderly  and  efficiently  integrate  and  interact. 

2 

There  is  some  level  of  specificity  to  characterize  the  Interaction  (i.e.,  ability 
to  influence)  between  technologies  through  their  interface. 
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An  Interface  between  technologies  has  been  identified  with  sufficient  detail 
to  allow  characterization  of  the  relationship. 


IRL  is  a  systematic  measurement  of  the  interfacing  of  compatible  interactions  for  various 
technologies  and  the  consistent  comparison  of  the  maturity  between  integration  points.  The 
introduction  of  IRL  to  the  assessment  process  not  only  checks  the  place  of  technology  on  an 
integration  readiness  scale,  but  also  presents  a  direction  for  improving  integration  with  other 
technologies.  Just  as  TRL  has  been  used  to  assess  the  risk  associated  with  developing 
technologies,  IRL  is  designed  to  assess  the  risk  associated  with  integrating  these  technologies. 
Now  that  both  the  technologies  and  integration  elements  can  be  assessed  and  mapped  along 
an  objective  numerical  scale,  the  next  challenge  is  to  develop  a  metric  that  can  assess  the 
maturity  of  the  entire  system  that  is  under  development.  Sauser,  Ramirez-Marquez,  Henry,  and 
DiMarzio  (2008)  were  able  to  demonstrate  how  using  a  normalized  matrix  of  pair-wise 
comparisons  of  TRLs  and  IRLs  for  any  system  under  development  can  yield  a  measure  of 
system  maturity,  called  Systems  Readiness  Level  (SRL).  The  SRL  metric  can  be  used  to 
determine  the  maturity  of  a  system  and  its  status  within  a  developmental  lifecycle.  Table  2 
presents  the  definitions  of  the  various  levels  of  the  SRL  and  a  representation  of  how  the  SRL 
index  correlates  to  a  systems  engineering  lifecycle. 

Table  2.  System  Readiness  Levels 


SRL 

Acquisition  Phase 

Definitions 

0.90  to 
1.00 

Operations  &  Support 

Execute  a  support  program  that  meets  operational 
support  performance  requirements  and  sustains 
the  system  in  the  most  cost-effective  manner  over 
its  total  lifecycle. 

0.70  to 
0.89 

Production 

Achieve  operational  capability  that  satisfies  mission 
needs. 

0.60  to 
0.79 

System  Deveiopment 
&  Demonstration 

Develop  system  capability  or  (increments  thereof); 
reduce  integration  and  manufacturing  risk;  ensure 
operational  supportability;  reduce  logistics  footprint; 
implement  human  systems  integration;  design  for 
production;  ensure  affordability  and  protection  of 
critical  program  information;  and  demonstrate 
system  integration,  interoperability,  safety  and 
utility. 

0.40  to 
0.59 

Technology 

Development 

Reduce  technology  risks  and  determine 
appropriate  set  of  technologies  to  integrate  into  a 
full  system. 

0.10  to 
0.39 

Concept  Refinement 

Refine  initial  concept;  develop  system/technology 
strategy. 

Note:  These  ranges  have  been  derived  conceptually  and  are  undergoing  field  verification  and 
validation  under  Naval  Postgraduate  School  Contract  #  N00244-08-0005. 


2.  Calculating  System  Readiness  Level 

The  computation  of  the  SRL  is  a  function  of  two  matrices: 
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1 .  Matrix  TRL  provides  a  blueprint  of  the  state  of  the  system  with  respect  to  the  readiness 
of  its  technologies.  That  is,  TRL  is  defined  as  a  vector  with  n  entries  for  which  the 
entry  defines  the  TRL  of  the  technology. 

2.  Matrix  IRL  illustrates  how  the  different  technologies  are  integrated  from  a  system 
perspective.  IRL  defined  as  an  nxn  matrix  for  which  the  element  IRL^  represents  the 
maturity  of  integration  between  the  /^and  technologies. 

In  these  matrices,  the  standard  TRL  and  IRL  levels  corresponding  to  values  from  1 
through  9  should  be  normalized.  Also,  it  has  been  assumed  that  on  the  one  hand,  a  value  of  0 
for  element  IRL^  defines  that  the  /“"^and technologies  are  impossible  to  integrate.  On  the  other 
hand,  a  value  of  1  for  element  IRL^  can  be  understood  as  one  of  the  following,  with  respect  to 
the  /'^and  f"  technologies:  1 )  are  completely  compatible  within  the  total  system,  2)  do  not 
interfere  with  each  others  functions,  3)  require  no  modification  of  the  individual  technologies, 
and  4)  require  no  integration  linkage  development.  Also,  it  is  important  to  note  that  IRL#  may 
have  a  value  lower  than  1 ,  Illustrating  that  the  technology  may  be  a  composite  of  different  sub¬ 
technologies  that  are  not  absolutely  mature. 

In  any  system,  each  of  the  constituent  technologies  is  connected  to  a  minimum  of  one 
other  technology  through  a  bi-directional  integration.  How  each  technology  is  integrated  with 
other  technologies  is  used  to  formulate  an  equation  for  calculating  SRL  that  is  a  function  of  the 
TRL  and  IRL  values  of  the  technologies  and  the  interactions  that  form  the  system.  In  order  to 
estimate  a  value  of  SRL  from  the  TRL  and  IRL  values,  we  propose  a  normalized  matrix  of  pair¬ 
wise  comparison  of  TRL  and  IRL  indices.  That  is,  for  a  system  with  n  technologies,  we  first 
formulate  a  TRL  matrix,  labeled  [TRL].  This  matrix  is  a  single  column  matrix  containing  the 
values  of  the  TRL  of  each  technology  in  the  system.  In  this  respect,  [TRL]  is  defined  in  Equation 
1 ,  where  TRLj  is  the  TRL  of  technology  /. 


Equation  1. 


[TRL  U 


TRL, 

TRL^ 


TRL 


n 


Second,  an  IRL  matrix  is  created  as  a  symmetric  square  matrix  (of  size  nxn)  of  all 
possible  integrations  between  any  two  technologies  in  the  system.  For  a  system  with  n 
technologies,  [IRL]  is  defined  in  Equation  2,  where  IRLy  is  the  IRL  between  technologies  /  and  j. 
It  Is  important  to  note  that  whenever  two  technologies  are  not  planned  for  integration,  the  IRL 
value  assumed  for  these  specific  technologies  is  the  hypothetical  integration  of  a  technology  /  to 
itself;  therefore,  it  is  given  the  maximum  level  of  9  and  is  denoted  by  IRU 


'iRLn 

IRL^^  . 

..  IRL 

Equation  2. 

[LRLtn  - 

IRL^^ 

LRL22 

..  IRL. 

_IRL„, 

iRL„2  • 

..  IRL, 

Although  the  original  values  for  both  TRL  and  IRL  can  be  used,  the  use  of  normalized 
values  allows  a  more  accurate  comparison  when  comparing  the  use  of  competing  technologies. 
Thus,  the  values  used  in  [TRL]  and  [IRL]  are  normalized  (0,1)  from  the  original  (1,9)  levels. 
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Based  on  these  two  matrices,  an  SRL  matrix  is  obtained  by  obtaining  the  product  of  the  TRL 
and  IRL  matrices,  as  shown  in  Equation  3. 

Equation  3.  Li  =  [iRL  x  [tRL 

The  SRL  matrix  consists  of  one  element  for  each  of  the  constituent  technologies.  From 
an  integration  perspective,  it  quantifies  the  readiness  level  of  a  specific  technology  with  respect 
to  every  other  technology  in  the  system  while  also  accounting  for  the  development  state  of  each 
technology  through  TRL.  Mathematically,  for  a  system  with  n  technologies,  [SRL]  is  as  shown  in 
Equation  4. 


IRL^J'RL^  +IRL^^TRL^  + ...  + IRL^JRL^ 
/RZ/2[77?Lj  +  IRL22TRL2  IRL2nTRL„ 

IRL„,TRL,  +IRL„2TRL2  +...  +  IRLTRL„ 

nl  1  nz  2  nn  n  _ 

where  IRLij=IRLji. 

Each  of  the  SRL  values  obtained  in  Equation  4  would  fall  within  the  interval  (0,n).  For 
consistency,  these  values  of  SRL  should  be  divided  by  n  to  obtain  the  normalized  value 
between  (0,1).  Notice  that  [SRL]  can  be  used  as  a  decision-making  tool  since  its  elements 
provide  a  prioritization  guide  of  the  system’s  technologies  and  integrations.  Thus,  [SRL]  can 
point  out  deficiencies  in  the  maturation  process. 

The  SRL  for  the  complete  system  is  the  average  of  all  such  normalized  SRL  values,  as 
shown  in  Equation  5.  Equal  weights  are  given  to  each  technology,  and  hence,  a  simple  average 
is  estimated.  A  standard  deviation  can  also  be  calculated  to  indicate  the  variation  in  the  system 
maturity  and  parity  in  subsystem  development. 


Equation  4. 


[srl]= 


SRL^ 

SRL, 


SRL„ 


Equation  5. 


^SRL,  SRL,  SRL  ^ 


SRL 


 "1 


‘■n  J 


where  ni  is  the  number  of  integrations  with  technology  /. 


3.  An  Example  of  SRL  Calculation 

The  following  example  will  use  a  real  blue-water  ship  that  is  currently  under  development 
to  show  the  steps  involved  in  calculating  its  SRL  value.  This  system  example  will  be  referred  to 
as  System  X  and  its  contemplated  architecture  is  shown  in  Figure  1 .  For  this  system,  the 
following  matrices  can  be  created  for  the  TRL  and  IRL,  based  on  the  definitions  presented 
earlier  in  Tables  1  and  2. 
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Figure  1.  Schematic  Architecture  of  System  X 


Equation  1a. 

[TRL^' 

TRL^ 


=[9  997699769987687689  9] 
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Equation  2a. 
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As  indicated  in  the  above  integration  matrix,  we  assign  an  IRL  value  of  0  when  there  is 
no  integration  link  contemplated  between  any  2  technologies.  For  integration  to  itself,  an  IRL 
value  of  9  is  used.  After  normalization  of  the  [TRL]  and  [IRL]  matrices,  calculate  [SRL]  as 
follows: 


Equations  3a  and  4a. 


[^^]=[^^]20x20[^]2M 


SRL, 

SRL, 


SRL,), 


Table  3.  Individual  SRL  Values 


SRLi 

SRL2 

SRL3 

SRL4 

SRLs 

SRLs 

SRL7 

SRLs 

SRLg 

SRL10 

2.000 

3.691 

2.605 

4.481 

1.963 

3.728 

2.000 

2.333 

2.000 

1.519 

SRL11 

SRL12 

SRL13 

SRL14 

SRL15 

SRL16 

SRL17 

SRL18 

SRL19 

SRL2O 

1.556 

1.444 

1.333 

1.481 

1.568 

5.778 

2.358 

2.099 

2.210 

1.519 

Equation  5a.  Composite  SRL 


^SRL,  SRL,  SRL  ^ 
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=  0.763 
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The  calculated  Composite  SRL  index  indicates  that  the  system  under  development  is 
currently  in  the  System  Development  and  Demonstration  phase.  Aside  from  providing  an 
assessment  of  overall  system  development,  it  can  also  be  a  guide  in  prioritizing  potential  areas 
that  require  further  development.  This  new  index  can  then  interact  with  decision-making  tools  for 
the  potential  acquisition  of  systems,  which  involve  the  dependency  and  interplay  among 
performance,  availability  (reliability,  maintainability,  and  supportability),  process  efficiency 
(system  operations,  maintenance,  and  logistics  support),  system  lifecycle  cost,  and  system 
maturity  (measured  by  SRL).  The  overarching  perspective  of  this  methodology  provides  a 
context  for  the  “trade  space”  available  to  a  systems  engineer  or  program  manager,  along  with 
the  articulation  of  the  overall  objective  of  maximizing  the  operational  effectiveness  of  systems. 

4.  Potential  Applications  of  SRL:  Future  Research 

Given  the  ability  to  estimate  readiness  of  a  system  under  development,  organizations 
can  systematically  evaluate  the  implications  of  using  alternative  technologies  or  system 
architectures,  prepare  development  plans  that  optimize  the  objectives  of  the  development  team, 
and  eventually  be  able  to  evaluate  and  monitor  the  progress  of  the  development  effort  to  identify 
problem  areas  and  corrective  measures. 

4.1  Optimization  Models 

In  the  defense  acquisition  process,  there  are  factors  that  may  strategically  alter  the 
decision  to  develop  one  system  over  another;  supersede  a  new,  more  functional  system  over 
another;  determine  if  a  system  or  technology  has  become  inadequate  due  to  changes  in  other 
systems  or  technologies;  invest  in  the  development  of  a  new  system  or  maintain  existing 
systems;  and  classify  a  systems  obsolescence  and  longevity.  To  address  these  challenges,  we 
can  use  SRL  as  a  method  for  determining  current  and  future  readiness  of  a  system  in  order  to 
determine  its  position  in  the  defense  acquisition  process.  While  identifying  the  current  SRL  of  a 
system  can  provide  managerial  insight,  optimizing  the  future  value  of  this  index  based  on 
constrained  resources  will  enhance  managerial  capabilities. 

The  optimization  of  SRL  based  on  resource  allocation  can  allow  for  decisions  to  be 
made  regarding  the  trade-offs  among:  1)  system  attributes  such  as  availability,  performance, 
efficiency,  and  total  ownership  cost  and  2)  the  components  necessary  for  producing  affordable 
system  operational  effectiveness  (pp.  14-15).  These  attributes  have  objectives  and  ranges  for 
components  such  as  capability,  reliability,  maintainability,  supportability,  and  producibility,  and  it 
is  the  interplay  among  them  that  drives  the  different  levels  for  both  IRL  and  TRL  of  the  elements 
in  a  system.  Thus,  the  optimal  selection  of  which  components  to  enhance  to  improve  the  system 
SRL  becomes  an  optimal  system  design  development  problem. 

The  optimal  design  of  systems  is  a  classical  optimization  problem  in  the  area  of  systems 
engineering.  In  general,  the  objective  of  these  problems  is  to  optimize  a  function-of-merit  of  the 
system  design  (reliability,  cost,  mean  time  to  failure,  supportability,  etc.)  subject  to  known 
constraints  on  resources  (cost,  weight,  volume,  etc.)  and/or  system  performance  requirements 
(reliability,  availability,  mean  time  to  failure,  etc.).  To  optimize  this  specific  function,  it  is 
generally  assumed  that  the  system  can  be  decomposed  into  a  system  that  contains  a  known 
number  of  subsystems  or  elements  (as  in  Figure  1 )  and,  for  each  of  these  elements,  a  known 
set  of  functionally  equivalent  components  types  (with  different  performance  specifications)  can 
be  used  in  the  design. 
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From  a  system  engineering  design  perspective,  an  optimization  approach  that  balances 
needs  (i.e.,  the  enhancement  of  the  SRL)  with  resources  (i.e.,  cost  of  technologies,  cost  of 
technology  development,  etc.),  can  be  an  effective  and  efficient  method  for  reducing  risk.  That 
is,  the  development  of  a  SRL  index  correlated  with  the  defense  acquisition  process  can  be  used 
as  an  optimization  framework  for  the  systems  engineer  or  program  manager  to  design-in 
enhanced  system  reliability,  maintainability,  and  supportability  to  achieve  the  desired  reductions 
in  the  necessary  logistics  footprint  and  the  associated  lifecycle  cost. 

Optimization  becomes  crucial  when  trying  to  decide  between  competing  system  design 
alternatives  or  when  trying  to  decide  which  individual  TRL  or  IRL  to  improve.  To  make  the  best 
decision,  optimization  models  can  be  developed  to  assist  management  to  choose  SRL 
improvement  opportunities.  It  is  reasonable  to  assume  that  improvements  will  result  in  costs 
associated  with  the  purchase  of  new  technology,  rework  of  existing  equipment,  training  of 
employees,  hiring  new  employees,  and  enhancements  to  information  technology  infrastructure. 
Two  models  can  be  developed.  The  first  model  considers  minimizing  the  development  cost 
associated  to  increasing  SRL  to  some  predefined  user  level,  X.  The  second  model  is  to 
maximize  the  SRL  (a  function  of  TRL  and  IRL)  under  constraints  associated  with  resources. 

The  mathematical  forms  of  these  models  follow. 

4.1.1  System  Cost  of  Development  (SCOD)  Minimization 

Model  SCODmin  illustrates  an  optimization  model  whose  objective  is  to  minimize 
development  cost  (a  function  of  TRL  and  IRL  development)  under  constraints  associated  with 
schedule  and  the  required  SRL  value.  The  general  mathematical  form  of  Model  SCODmin 
follows: 

Minimize:  SCOD(TRL,IRL)  =  SCODfixed  +  SCODvariable  (TRL, IRL) 

Subject  to:  SRL(TRL,IRL)  >  A 
R1  (TRL, IRL)  <  r1 


Rh  (TRL,IRL)  <  rh 

The  matrices  IRL  and  TRL  in  Model  SCODmin  contain  the  decision  variables.  Each  of 
these  variables  are  integer  valued  and  bounded  by  {IRL,, 9)  and  {TRL,, 9),  respectively.  That  is, 
the  TRL/IRL  for  the  component  cannot  be  below  its  current  level  or  above  perfect  technology 
development/integration  (IRL  or  TRL  =  9). 

To  completely  characterize  the  decision  variables  in  Model  SCODmin,  it  is  necessary  to 
introduce  the  following  transformation: 
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Notice  that  based  on  these  binary  variables,  each  of  the  possible  TRL  and  IRL  in  the 
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system  can  be  obtained  as  TRL.  =  and  IRL^  =  '^kxy .  Based  on  these  binary  variables, 
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Thus,  based  on  the  computation  of  the  SRL  with  these  decision  variables.  Model 
SCODmin  belongs  to  the  class  of  binary,  integer-valued,  non-linear  problems.  For  a  system  with 
n  technologies  containing  m  {m<{n-^  )n/2)  distinct  integrations,  and  assuming  all  technologies 
and  integrations  are  at  their  lowest  levels,  there  are  potential  solutions  to  Model  SCODmin- 
Evaluating  each  possible  solution  is  prohibitive,  so  to  generate  an  optimal  solution  faster,  a 
meta-heuristic  approach  developed  by  Ramirez-Marquez  and  Rocco  (Ramirez-Marquez  & 
Rocco,  2008)  will  be  applied  to  the  system  under  development.  This  approach,  called 
Probabilistic  Solution  Discovery  Algorithm  (PSDA),  has  the  capability  of  producing  quasi-optimal 
solutions  in  a  relatively  short  period  of  time.  However,  it  must  be  mentioned  that  the  results 
cannot  be  proven  to  be  the  optimal  solution.  Nevertheless,  prior  tests  have  indicated  that  PSDA 
results  tend  to  be  better  than  results  from  alternative  meta-heuristic  approaches. 


4.1.2  SRL  Maximization 

Model  SRLmax  follows  the  same  general  formulation.  It  illustrates  the  optimization  model 
with  the  objective  to  maximize  the  SRL  (a  function  of  TRL  and  IRL)  under  constraints  associated 
with  resources.  This  model  recognizes  that  the  technologies  compete  for  resources  and  that 
benefits  can  result  in  an  improved  SRL  via  the  optimal  allocation  of  such  resources.  The  general 
mathematical  form  of  Model  SRLmax  is 


Model  SRLmav 

Max  SRL  (TRL  ,IRL  ) 
s.t. 

i?,(TRL  ,IRL  )< 


i?^(TRL  ,IRL  )< 

The  success  of  implementing  these  models  depends  on  the  consistent  and  continuous 
definition  of  needed  capabilities,  the  maturation  of  technologies  that  lead  to  disciplined 
development,  and  the  production  of  systems  that  provide  increasing  capability  towards  a 
material  concept.  A  fundamental  challenge  to  defense  acquisition  is  that  the  ultimate 
functionality  cannot  be  defined  at  the  beginning  of  the  program.  Only  by  the  maturation  of  the 
technologies,  matched  with  the  evolving  needs  of  the  user,  can  they  provide  the  user  with 
capability. 
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4.2  System  Earned  Readiness  Management  (SERM) 

The  optimization  models  above  can  provide  valuable  insight  into  the  development  of  a 
methodology  for  monitoring  and  evaluating  the  overall  progress  of  the  development  effort.  This 
is  primarily  due  to  the  fact  that  the  models  can  identify  the  optimal  development  path  that  can  be 
followed.  That  is,  they  identify  to  what  TRL  the  critical  technology  elements  (CTEs)  and  which 
IRL  the  integration  elements  should  be  matured,  as  well  as  when  those  TRLs  and  IRLs  can  be 
achieved. 

With  these  data,  we  can  develop  an  analytical  tool  and  methodology  for  evaluating 
overall  progress  in  systems  development  as  well  as  measure  the  impact  of  alternative  or 
competing  architectures,  critical  technologies  and  integration  elements  on  the  maturity  of 
systems  within  the  systems  engineering  lifecycle.  Furthermore,  it  can  serve  as  a  guide  to 
anticipate  the  lifecycle  implications  of  the  decisions  made  during  the  development  process.  The 
proposed  methodology  is  termed  System  Earned  Readiness  Management  (SERM).  It  will  be 
analogous  to  Earned  Value  Management  (EVM),  an  analytical  tool  used  in  Project  Management 
(pp.  17-18). 

While  the  optimization  models  are  unavoidably  mathematically  involved,  SERM  itself  is 
envisioned  to  be  a  relatively  simple  management  tool.  It  will  measure  in  aggregate  terms  the 
level  of  accomplishment  of  the  system  development  process.  When  compared  to  the 
development  plans  and  factor  estimates  that  have  been  prescribed  for  a  particular  system  under 
development,  management  can  make  conclusions  on  its  status  and  suggest  necessary 
adjustments  to  correct  any  significant  deviations.  SERM  is  expected  to  be  valid  throughout  a 
wide  range  of  systems  with  varying  degrees  of  complexity  and  is  intended  to  be  a  tool  that  is 
easy  to  use,  notwithstanding  the  complex  mathematical  algorithms  behind  it. 

Logically,  SERM  can  only  be  useful  if  the  system  under  development  is  already  covered 
by  a  sufficiently  detailed  development  plan.  That  is,  the  system  requirements,  design  and 
development  schedules  have  already  been  frozen.  However,  there  are  many  systems  under 
development  that  are  inherently  fraught  with  high  degrees  of  uncertainty  that  emanate  from  the 
high  levels  of  novelty  as  well  as  technology  of  the  system.  To  be  properly  managed,  such 
systems  have  to  go  through  several  requirements  and  design  cycles  before  both  can  be  frozen 
(Shenhar  &  Dvir,  2007).  However,  the  need  for  monitoring  and  evaluating  these  systems  before 
the  final  development  cycle  still  exists.  Developing  a  modified  SERM  (to  be  called  SERM-U)  for 
such  situations  will  be  the  ultimate  objective. 

5.  Conclusions 

This  paper  proposes  the  inclusion  of  a  separate  maturity  index  to  measure  the  progress 
of  the  development  of  the  integration  links  of  a  system  under  development.  This  metric  called 
Integration  Readiness  Level  (IRL)  is  necessary  because  in  some  projects,  integration  elements 
have  been  overlooked  and  have  resulted  into  major  debacles.  The  paper  also  introduces  the 
development  of  a  system-focused  approach  for  managing  system  development  and  making 
effective  and  efficient  decisions  during  the  defense  acquisition  process.  For  this  to  be 
accomplished,  a  new  System  Readiness  Level  (SRL)  index  will  incorporate  both  the  current 
TRL  scale  and  the  proposed  IRL  metric.  The  foundations  of  the  SRL  are  described  and  we 
show  the  techniques  for  determining  current  and  future  readiness  of  a  system  to  determine  its 
position  in  the  defense  acquisition  process.  In  addition,  it  proposes  optimization  models  than 
can  provide  management  with  an  optimal  development  plan  that  can  meet  the  objectives  of  the 
development  team  based  on  constrained  resources.  These,  in  turn,  can  become  the  foundation 
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for  the  development  of  a  monitoring  and  evaluation  tool  that  will  be  analogous  to  Earned  Value 
Management,  which  is  used  in  project  management. 


The  conceptual  development  of  these  metrics  and  tools  outpace  their  validation  and 
verification  in  the  field.  Currently,  what  is  necessary  is  to  have  greater  involvement  from 
practitioners  so  that  the  acquisition  community  can  agree  to  a  common  measurement  and 
language  that  can  only  improve  the  system  development  and  acquisition  process. 
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System  Maturity  Indices  for  Decision 


Support  in  the  Defense  Acquisition  Process 


Brian  J.  Sauser,  Ph.D.  and  Jose’  E.  Ramirez-Marquez,  Ph.D. 

Stevens  Institute  of  Technoiogy 


Abstract 


In  the  National  Aeronautics  and  Space  Administration  (NASA)  and  the 
Department  of  Defense  (DoD)  the  Technology  Readiness  Level  (TRL) 
scale  is  a  measure  of  maturity  of  an  individual  technology,  with  a  view 
towards  operational  use  in  a  system  context.  A  comprehensive  set  of 
concerns  becomes  relevant  when  this  metric  is  abstracted  from 
an  individual  technology  to  a  system  context,  which  may  involve 
interplay  among  multiple  technologies  that  are  integrated  through 
a  systems  engineering  process.  This  research  proposes  the 
development  of  a  system-focused  approach  for  managing  system 
development  and  making  effective  and  efficient  decisions  during  a 
systems  engineering  process.  This  research  will  present  a  System 
Readiness  Levei  (SRL)  index  that  incorporates  both  the  current 
TRL  scale  and  the  concept  of  an  Integration  Readiness  Level  (IRL) 
and  provide  a  method  for  determining  current  and  future 
readiness  of  a  system  to  determine  its  potentiai  position  in  the 
systems  engineering  process. 
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What’s  Missing  in  TRL? 


•  A  complete  representation  of  the  (difficulty  of)  integration  of  the 
subject  technology  or  subsystems  into  an  operational  system 
(Dowling  and  Pardoe,  2005,  Mankins,  2002,  Meystel  et  al., 
2003,  Smith,  2005,  Valerdi  and  Kohl,  2004), 


•  The  uncertainty  that  may  be  expected  in  moving  through  the 
maturation  of  TRL  (Shishko  et  al.,  2003,  Cundiff,  2003,  Dowling 
and  Pardoe,  2005,  Mankins,  2002,  Smith,  2005,  Moorehouse, 
2001),  and 


•  Comparative  analysis  techniques  for  alternative  TRLs  (Cundiff, 
2003,  Dowling  and  Pardoe,  2005,  Mankins,  2002,  Smith,  2005, 
Valerdi  and  Kohl,  2004). 


‘In  order  to  succeed  over  the  longer  term,  additional  methodologies  are  needed,  including  those  which  allow  the 
identification  of  anticipated  uncertainty  in  planned  R&T  programs ...  ”  (Mankins,  2002) 


Mtinlcrej’,  r  \ 
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other  Work 


•  Manufacturing  Readiness  Level  (DoD) 

-  Used  to  assess  the  SE/design  process  and  maturity  of  a 
technology’s  associated  manufacturing  processes  to  enable  rapid, 
affordable  transition  to  acquisition  programs. 

•  Integrated  Technology  Analysis  Methodology  (ITAM)  (Mankins, 
2002) 

-  Discipline-neutral,  quantitative  measure  of  the  relative  technological 
challenge  inherent  in  various  candidate/competing  advanced 
systems  concepts. 


•  Systems  Integration  Readiness  Level  (MoD) 


-  System  Readiness  Levels  (SRLs)  were  developed  as  a  tool  for 
projects  to  assess  System  Maturity,  and  to  communicate  this  in  a 
consistent  manner. 

•  Capability  RL,  Design  RL,  Habitation  RL,  Human  RL,  Logistics 
RL,  Operational  RL,  and  Software  RL 
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Why  do  we  need  a  Systems  Readiness 
Level  (SRL)? 


•  TRL  is  only  a  measure  of 
an  individual  technology: 
it  gives  no  indication  of  a 
systems  readiness. 

•  There  is  no  method  for 
integrating  TRLs 


L 


There  is  no  systematic 
measure  of  a  systems 
readiness. 


A  1999  GAO  Report  stated  that  programs  started 
with  a  technology  at  TRL  5  or  below  experienced 
“significant  cost  and  schedule  increases.”  GAO 
also  recommended  that  technologies  should 
mature  until  the  equivalent  of  TRL  7  before  they 
are  Included  In  weapon  system  programs. 


- = - - - - - ^~““JTITITITrTTl 
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Technology  Readiness  Level 


Parallel  (not  integrated)  Development 


Phase 


Operations  &  Support  (4.3.5) 

9 

Production  Development  (4.3.4) 

rs 

System  Development  & 
Demonstration  (4.3.3) 


Technology  Development  (4.3.2) 


Concept  Refinement  (4.3.1) 


Basic  Technology  Research 


Point  of  Transition  or 
Integration  * 
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Systems  Readiness  Level 


The  System 


SRL  =  f(TRL,  IRL) 


Development  of  metrics,  tool,  and  methodologies  for 
determining  a  systems  readiness  level  (SRL)  and 
potential  for  making  efficient  and  effective  life-cycle 
acquisition  and  operational  decisions.  The  SRL  Model 
is  a  function  of  the  individual  Technology  Readiness 
Levels  (TRL)  and  their  subsequent  integration  points 
with  other  technologies,  the  Integration  Readiness 
Level  (IRL). 


-  Value  Proposition: 

•  Currently  TRL  is  only  a  measure  of  an  individual 
technology 

•  There  is  no  method  for  integrating  TRLs 

•  There  is  no  systematic  measure  of  a  systems  readiness 

•  Cost  and  schedule  reduction  in  strategic  technology 
development  planning 

-  Deliverable:  Integration  of  methodologies  for 
strategic  roadmap  planning  that  illustrate  the 
timely  implementation  of  capability  increments. 
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Maturity  Indices 


TRL  Definition 

IRL  Definition 

SRL  Definition 

SRL  VALUE 

1 

Actual  system  proven  through 
successful  mission 
operations 

Integration  is  mission  proven  through 

Operations  and  Support 

0.90  to  1.00 

2 

successful  mission  operations 

Production 

0.80  to  0.89 

8  1 

Actual  system  completed  and 
qualified  through  test  and 
demonstration 

Actual  integration  completed  and  mission 
qualified  through  test  and  demonstration 
in  the  system  environment 

System  Development  and 

0.60  to  0.79 

7 

System  prototype 
demonstration  in  relevant 
environment 

The  integration  of  technologies  has  been 
verified  and  validated  with  sufficient  detail 
to  be  actionable 

Demonstration 

6 

System/subsystem  model 
demonstration  in  relevant 
environment 

The  integrating  technologies  can  accept, 
translate  and  structure  information  for  its 
intended  application 

Technology  Development 

0.40  to  0.59 

5 

Component  and/or 
breadboard  validation  in 
relevant  environment 

There  is  sufficient  control  between 
technologies  necessary  to  establish, 
manage  and  terminate  the  integration 

4 

Component  and/or 
breadboard  validation  in 
laboratory  environment 

There  is  sufficient  detail  in  the  quality  and 
assurance  of  the  integration  between 
technologies 

3 

Analytical  &  experimental 
critical  function  and/or 
characteristic  proof-of- 
concept 

There  is  compatibility  between 
technologies  to  orderly  and  efficiently 
integrate  and  interact 

Concept  Refinement 

0.10  to  0.39 

2 

i 

Technology  concept  and/or 
application  formulated 

There  is  some  level  of  specificity  to 
characterize  the  interaction  between 
technologies  through  their  interface 

1 

Basic  principles  observed 
and  reported 

An  interface  between  technologies  has 
been  identified  with  sufficient  detail  to 
allow  characterization  of  the  relationship 
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Step  1 :  Identify  hardware  and 
software  systems  to  be  analyzed 


Include  the  major  technologies  and 
components  that  make-up  the  overall 
system 


Step  2:  Define  network  diagram  for 
systems 


Emphasis  is  on  the  proper  depiction  of 
hardware  and  software  integration  between 
systems 


Step  3:  Apply  detailed  TRL  and  IRL 
evaluation  criteria  to  systems 


Step  4:  Calculate  individual  and 
composite  SRLs 


Step  5:  Document  status  via  roll-up 
charts 


Integration  Maturity  Level  1 

100  Components  to  be  integrated  are  selected 
100  Component  interface  points  are  identified 
100  Documented  phase/build  plan  for  component  availability 


inn  Datatupflt;  irlantifiarl 


Technology  Readiness  Level  1 


100  Physical  laws  and  assumptions  used  in  new  technologies  defined 
EH  Have  some  concept  in  mind  that  may  be  realizable  in  software 
Know  what  software  needs  to  do  in  general  terms 
Paper  studies  confirm  basic  principles  and  system  concepts 
Mathematical  formulations  of  concepts  that  might  be  realizable  in  software 
100  Have  an  idea  that  captures  the  basic  principles  of  a  possible  algorithm 
Basic  scientific  principles  observed 
100  Research  hypothesis  formulated 

Identify  who  will  perform  research  and  where  it  will  be  done 
60  Readiness  Level  Percent  Complete  (non-weighted) 


Utilize  detailed  evaluation  criteria  to 
determine  the  TRL  of  identified  systems  and 
the  IRL  of  defined  connections 


Leverage  TRL  and  IRL  evaluations  to 
compute  an  assessment  of  overall 
system  status  via  SRLs 


Populate  reporting  chart  templates 
with  evaluation  and  calculation 
outcomes  to  highlight  both  current 
^^^^ormance  over  time 


Iterative  SME  Evaluation  Throughout  Development  Cycle 
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System  Alpha 

Determining  the  TRL  and  iRL 


TRL.  =  9 


IRL, ,2  =  1 


Technology 


Technology 


TRL2=  6 


IRL2,3  =  7 


TRL3  =  6 
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System  Alpha  -  TRL 
Creating  the  TRL  Matrix 


TRL^  =  9 


IRL1.2  =  1 


TRL  Matrix 


r 

TRLi 

9 

TRL2 

— 

6 

TRL3 

6 

V.  V. 


TRL2  = 


Technology 

3 


'^TRL3=  6 


Mtinlcr^jv  \ 
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System  Alpha  -  IRL 
Creating  the  IRL  Matrix 


TRLi  =  9 


IRL  Matrix 


IRL, ,  =  1 


Technology 

1 


Technology 

2 


TRL2=  6 


'  IRL,  IRL, 2 

9 

1 

0 

IRL,  3 

— 

1 

9 

7 

IRL2,  IRL2 

7 

9 

IRL 


23 


No  Integratior 


Technology 

3 


IRL,, 3  =  7 


Integration  to  Self 


TRL3=  6 


f  r  ! 
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SRL  for  System  Alpha 
Normalizing  the  TRLs  and  IRLs 


^IRL^ 

IRL12 

IRL1? 

TRL^ 

IRL^i 

IRL2 

IRL23 

TRL2 

^IRL31 

IRL32 

IRL^ 

TRL3 

L  J 

^9  1  0^ 

r  ^ 

9 

1  9  7 

6 

0  7  9 

6 

L  J 

L. _ _ _ ^ ^ J 


Non-Normalized  [(0,9)  scale] 


r 

1.0 

0.11 

0.0^ 

1.0 

0.11 

1.0 

.78 

0.67 

0.0 

.78 

1.0^ 

0.67 

L  J 

SRL  Calculation  of  System  Alpha 
Calculating  the  SRLx 


SRL  =  I RL  X  TRL 


SRLi 

TrLi 

IRL12 

1  RLi3^ 

SRL2 

= 

IRLji 

IRL2 

IRL23 

SRL3 

^1-31 

IRL32 

irlJ 

r 

r 

SRLi 

1.07 

SRL2 

— 

1.30 

SRL3 

1.19 

J 

J 

Note:  SRL^  represents  Technology  X  and  its  I  RLs 


TRLi 

TRL2 

TRL3 

V. 


(0,nx)  scale 


SRL  for  System  Alpha 
Calculating  the  Composite  SRL 


r 


SRLi  SRLj  SRL3 


r 


1.07  1.30  1 


SRLi  SRL;  SRL^ 


r 


0.54  0.43  0.60 


(0,1)  scale 


Composite  SRL 


=  1/3  (0.54  +  0.43  +  0.60) 


0.52 


Items 


and  Systems  Engineering. 


-jrrr 


(fort 


coming) 


3(6) 


SRL  Mapping 


0.77  to  1 .00  Operations  & 
Support 


Execute  a  support  program  that  meets  operational  support 
performance  requirements  and  sustains  the  system  in  the  most 
cost-effective  manner  over  its  total  life  cycle. 


0.68  to  0.76  Production 


Achieve  operational  capability  that  satisfies  mission  needs. 


0.60  to  0.67  System 

Deveiopment  & 
Demonstration 


SRL3  =  0.60 


Develop  system  capability  or  (increments  thereof);  reduce 
integration  and  manufacturing  risk;  ensure  operational 
supportability;  reduce  logistics  footprint;  implement  human 
systems  integration;  design  for  production;  ensure  affordability 
and  protection  of  critical  program  information;  and  demonstrate 
system  integration,  interoperability,  safety  and  utility. 


SRL1  =  0.54  0.40  to  0.59 


SRLcomp  =  0.52 
SRL2  =  0.43 


0.10  to  0.39 


Technology 

Deveiopment 

Concept 

Deveiopment 


Reduce  technology  risks  and  determine  appropriate  set 
Refine  initial  concept;  Develop  system/technology  strategy. 
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NOTE:  ALL  DATA  IN  THIS  TEMPLATE  IS  NOTIONAL 

Detailed  Status 


ISS  C2MC  GMRCS 


SRL  O 


y 


GEP 


MEC 


A 


Composite  Actual 

Planned 

SRL 

.57 

.6 

SRL  w/o 

Sea  Frame 
Integrations 

.53 

.6 

□ 


O 

o 

• 

V 

V 

V 


LEGEND 

Mission  System 
Sea  Frame  System 

Current  Mission  Package  SRL  Status 
Previous  Mission  Package  SRL  Status 

Current  Mission  System  SRL  Status 
Technology  Readiness  Level 
Integration  Maturity  Level 

System  Readiness  Level  Demarcation 
Scheduled  Position 
Low  Risk  to  Cost  and/or  Schedule 

Moderate  Risk  to  Cost  and/or  Schedule 
High  Risk  to  Cost  and/or  Schedule 


Data  Collection  Period:  XX/XX/XX  -  X/XX/XX 
Previous  Report  Date:  XX/XX/XX 
Schedule  Updated:  09/25/07  (QER) 

NLOS  MK  46  NEP  GCCS-M 


■9 


■o- 


■o- 


NOTE:  ALL  DATA  IN  THIS  TEMPLATE  IS  NOTIONAL 


System  Status  Roll-up 


FY08 

FY09 

FYIO 

FYll 

Q1  Q2  Q3  Q4 

Q1  Q2  Q3  Q4 

Q1  Q2  Q3  Q4 

Q1  Q2  Q3  Q4 

n 

MIW 


ASW 


SUW 


■9- 


■9 


9 


0 


2  12/15/08 


Fleet  Assessment 


DT 


Jfr 


0 


9/30/08 


1  /  6/15/08 
*WSESRB 
6/6I0S 


Fleet  Assessment 


SW  Cert. 


5/15/09 


9 


Fleet  Assessment 


CDl^ 


0 


11/15/09 


OA 


9 


SW  Cert. 


8/15/09 


2>  6/15/09 


2 


0^ 


0 


DT 


2/15/10 


[i^ 

-o 


DT 


DT 


SW  Cert. 


6/15/10 


T 


0 


8/15/10  5  11/15/10 


OA 


OA 


OA 


[l^ 

o— 


3>  5/15/11 


0^ 


t 


LEGEND 

▼ 

Current  Reporting  Period  Status 

— 

Scheduled  Position 

Previous  Reporting  Period  Status 

o 

System  Readiness  Level 

Data  Collection  Period:  XX/XX/XX  -  X/XX/XX 
Previous  Report  Date:  XX/XX/XX 
Schedule  Updated:  09/25/07  (QER) 


Current  Research 


•  SRL  Resource  Optimization 

•  System  Earned  Readiness  Management 
(SERM) 

•  SRL  Confidence 

•  SRL  String  (“Theory”) 


Resource  Optimization  Models  and 
System  Earned  Readiness 
Management  (SERM) 


ROBOTIC 
SERVICING 
MISSION  FOR 
THE  HUBBLE 
SPACE 
TELESCOPE 


Tech  1-  Remote  Manipulator  System  (RMS); 

Tech  2  -  Special  Purpose  Dexterous  Manipulator  (SPDM); 
Tech  3  -  Electronic  Control  Unit  (ECU); 

Tech  4  -  Autonomous  Grappling  (AG); 

Tech  5  -  Autonomous  Proximity  Operations  (APO); 
Tech  6  -  Laser  Image  Detection  and  Radar  (LIDAR). 
CURRENT  SRL  =  0.48  (still  in  Technology  Development) 
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SRL  Resource  Optimization 


Model  SRLn,ax  =  optimization  model 
with  the  objective  to  maximize  the  SRL  (a 
function  of  TRL  and  IRL)  under  constraints 
associated  with  resources. 


Case 

SRLl 

SRL2 

SRL3 

SRL4 

SRL5 

SRL6 

SRL 

COST, 

Smillion 

TIME, 

man-hours 

100% 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

26.574 

19,122 

75% 

0.96 

0.94 

0.96 

0.89 

0.86 

1.00 

0.93 

19.892 

14,044 

60% 

0.96 

0.92 

0.94 

0.78 

0.78 

0.89 

0.90 

15.870 

10,254 

45% 

0.93 

0.84 

0.89 

0.67 

0.71 

0.83 

0.81 

11.930 

7,283 

30% 

0.89 

0.73 

0.84 

0.52 

0.64 

0.78 

0.73 

7.727 

4,961 

15% 

0.76 

0.66 

0.76 

0.46 

0.56 

0.67 

0.64 

3.991 

2,733 

Current  Status 

0.48 
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SRL  Resource  Optimization 


Model  SCOD^in  =  an  optimization  model 
whose  objective  is  to  minimize  development 
cost  (a  function  of  TRL  and  IRL 
deveiopment)  under  constraints  associated 
with  scheduie  and  the  required  SRL  value. 


Desired  Improvements 

SRL 

Time  (man-hrs) 

Computed  Minimum 

in  SRL  (%) 

Targeted 

Computed 

Targeted 

Computed 

Cost  ($  x1000) 

0 

0.480 

0.480 

n.a 

n.a 

n.a 

20 

0.584 

0.587 

3,824 

1,654 

2,203 

40 

0.688 

0.692 

7,649 

3,797 

5,914 

60 

0.792 

0.794 

11,473 

7,667 

11,065 

80 

0.896 

0.896 

15,298 

11,309 

16,888 

AcqulslLiaa  Research  Prpgrum:  Creating  Synetg}'  for  Informed  Chariige 


Design  Solution  from  SCODmin  Model 


Target 

SRL 

TRL 

IPIL 

1 

2 

3 

4 

5 

6 

1,2 

1,3 

2,3 

2,4 

3,5 

4,5 

5,6 

1 

(Year  5) 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

0.896 

(Year  4) 

9 

9 

9 

8 

9 

9 

9 

9 

9 

8 

8 

5 

7 

0.792 

(Year  3) 

8 

9 

9 

6 

9 

9 

9 

9 

9 

5 

8 

4 

6 

0.688 

(Year  2) 

8 

8 

9 

6 

9 

9 

8 

8 

7 

5 

7 

2 

4 

0.584 

(Year  1) 

8 

8 

8 

6 

7 

6 

7 

7 

7 

5 

6 

2 

4 

0.48 

(Year  0) 

8 

8 

7 

6 

6 

6 

5 

6 

6 

5 

6 

2 

2 
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Systems  Earned  Readiness  Management 


Managerial  Intervention 


Performance  Measures  / 
Variances 


System  Development 
Schedules 


Optimal  Design  Solutions 


Systems  Readiness  Level 


Constrained  Optimization 
Models 


SRL  Confidence 


SRL  Confidence 


Context  Diagram 


TRL  &  IRL  States  Table 


Technology  (i) 

TRL{k) 

Probability  {po  k) 

Integrationiij)  IRL(k) 

Probability  (gm) 

1 

7 

8 

9 

0.15 

0.80 

0.05 

1,2 

4 

5 

6 

0.05 

0.60 

0.35 

2 

7 

8 

9 

0.05 

0.80 

0.15 

13 

5 

6 

7 

0.20 

0.70 

0.10 

3 

6 

7 

8 

0.10 

0.80 

0.10 

23 

5 

6 

7 

0.05 

0.80 

0.15 

4 

5 

6 

7 

0.15 

0.80 

0.05 

2,4 

4 

5 

6 

0.10 

0.80 

0.10 

5 

5 

6 

7 

0.25 

0.70 

0.05 

3,5 

5 

6 

7 

0.20 

0.70 

0.10 

6 

5 

6 

7 

0.20 

0.70 

0.10 

4,5 

1 

2 

3 

0.15 

0.60 

0.25 

5,6 

2 

3 

0.40 

0.60 
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SRL  Confidence  Simulation 


Composite  SRL 


Certainty  is  90.00%  from  0.478  to  0.  537 
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SRL  Confidence  Simulation  Results 


Percentile 

SRL 

SRL1 

SRL2 

SRL3 

SRL4 

SRL5 

SRL6 

0% 

0.435 

0.514 

0.448 

0.475 

0.325 

0.278 

0.340 

5% 

0.478 

0.584 

0.534 

0.556 

0.391 

0.333 

0.352 

25% 

0.495 

0.617 

0.562 

0.577 

0.416 

0.358 

0.395 

50% 

0.507 

0.634 

0.574 

0.602 

0.436 

0.370 

0.426 

75% 

0.519 

0.667 

0.593 

0.617 

0.457 

0.389 

0.444 

95% 

0.537 

0.691 

0.620 

0.645 

0.481 

0.417 

0.481 

100% 

0.587 

0.786 

0.682 

0.719 

0.556 

0.488 

0.519 

N:iV;il  ficJiCKJ-l 
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SRL  Confidence  Analysis 


AcqulslLiaa  Research  Prpgrum:  Creeling  Synetg}'  for  Informed  ChoHige 


Mtinlcrej’,  r  \ 


SRL  String  (“Theory”) 


SRL  String  (“Theory”) 


•  SRL  Evaluations  per  Mission  Thread 

•  Automatically  weights  those  technologies 
most  important  to  the  system 
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Future  Research  Plans 


•  Case  studies  for  SRL  Mapping  to  Life  Cycles 

-  New  projects.  Moving  forward 

-  Historical  data.  Failed  projects.  Completed  projects 

•  SERM  Toolset 

-  Identify  SERM  Toolset,  i.e:  algorithms,  applications 

-  Evaluate  toolset  with  case  studies 


•  Forecasting  and  Road  mapping 

-  Applications  for  predictive  cost  and  risk  forecasting  with  business 
case  analysis 

-  SRL  calculator  with  architecture  formation 

-  Technology  tradeoff  environment 

-  Disruptive  Technologies  in  Systems  Maturity  Forecasting 

-  Vendor  Selection  in  System  Maturity  Assessment 
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